Learning to reach and reaching to learn: a unified approach to path planning and reactive control through reinforcement learning

The next generation of intelligent robots will need to be able to plan reaches. Not just ballistic point to point reaches, but reaches around things such as the edge of a table, a nearby human, or any other known object in the robot’s workspace. Planning reaches may seem easy to us humans, because we do it so intuitively, but it has proven to be a challenging problem, which continues to limit the versatility of what robots can do today. In this document, I propose a novel intrinsically motivated RL system that draws on both Path/Motion Planning and Reactive Control. Through Reinforcement Learning, it tightly integrates these two previously disparate approaches to robotics. The RL system is evaluated on a task, which is as yet unsolved by roboticists in practice. That is to put the palm of the iCub humanoid robot on arbitrary target objects in its workspace, start- ing from arbitrary initial configurations. Such motions can be generated by planning, or searching the configuration space, but this typically results in some kind of trajectory, which must then be tracked by a separate controller, and such an approach offers a brit- tle runtime solution because it is inflexible. Purely reactive systems are robust to many problems that render a planned trajectory infeasible, but lacking the capacity to search, they tend to get stuck behind constraints, and therefore do not replace motion planners. The planner/controller proposed here is novel in that it deliberately plans reaches without the need to track trajectories. Instead, reaches are composed of sequences of reactive motion primitives, implemented by my Modular Behavioral Environment (MoBeE), which provides (fictitious) force control with reactive collision avoidance by way of a realtime kinematic/geometric model of the robot and its workspace. Thus, to the best of my knowledge, mine is the first reach planning approach to simultaneously offer the best of both the Path/Motion Planning and Reactive Control approaches. By controlling the real, physical robot directly, and feeling the influence of the con- straints imposed by MoBeE, the proposed system learns a stochastic model of the iCub’s configuration space. Then, the model is exploited as a multiple query path planner to find sensible pre-reach poses, from which to initiate reaching actions. Experiments show that the system can autonomously find practical reaches to target objects in workspace and offers excellent robustness to changes in the workspace configuration as well as noise in the robot’s sensory-motor apparatus

Spin-Forster transfer in optically excited quantum dots

The mechanisms of energy and spin transfer in quantum dot pairs coupled via the Coulomb interaction are studied. Exciton transfer can be resonant or phonon-assisted. In both cases, the transfer rates strongly depend on the resonance conditions. The spin selection rules in the transfer process come from the exchange and spin-orbit interactions. The character of energy dissipation in spin transfer is different than that in the traditional spin currents. The spin-dependent photon cross-correlation functions reflect the exciton transfer process. In addition, a mathematical method to calculate F\"orster transfer in crystalline nanostructures beyond the dipole-dipole approximation is described.Comment: 22 pages, 10 figures, Phys. Rev. B, in pres

Towards adaptive and autonomous humanoid robots: from vision to actions

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions

Better than a lens -- Increasing the signal-to-noise ratio through pupil splitting

Lenses are designed to fulfill Fermats principle such that all light interferes constructively in its focus, guaranteeing its maximum concentration. It can be shown that imaging via an unmodified full pupil yields the maximum transfer strength for all spatial frequencies transferable by the system. Seemingly also the signal-to-noise ratio (SNR) is optimal. The achievable SNR at a given photon budget is critical especially if that budget is strictly limited as in the case of fluorescence microscopy. In this work we propose a general method which achieves a better SNR for high spatial frequency information of an optical imaging system, without the need to capture more photons. This is achieved by splitting the pupil of an incoherent imaging system such that two (or more) sub-images are simultaneously acquired and computationally recombined. We compare the theoretical performance of split pupil imaging to the non-split scenario and implement the splitting using a tilted elliptical mirror placed at the back-focal-plane (BFP) of a fluorescence widefield microscope

Bovine polledness

The persistent horns are an important trait of speciation for the family Bovidae with complex morphogenesis taking place briefly after birth. The polledness is highly favourable in modern cattle breeding systems but serious animal welfare issues urge for a solution in the production of hornless cattle other than dehorning. Although the dominant inhibition of horn morphogenesis was discovered more than 70 years ago, and the causative mutation was mapped almost 20 years ago, its molecular nature remained unknown. Here, we report allelic heterogeneity of the POLLED locus. First, we mapped the POLLED locus to a ∼381-kb interval in a multi-breed case-control design. Targeted re-sequencing of an enlarged candidate interval (547 kb) in 16 sires with known POLLED genotype did not detect a common allele associated with polled status. In eight sires of Alpine and Scottish origin (four polled versus four horned), we identified a single candidate mutation, a complex 202 bp insertion-deletion event that showed perfect association to the polled phenotype in various European cattle breeds, except Holstein-Friesian. The analysis of the same candidate interval in eight Holsteins identified five candidate variants which segregate as a 260 kb haplotype also perfectly associated with the POLLED gene without recombination or interference with the 202 bp insertion-deletion. We further identified bulls which are progeny tested as homozygous polled but bearing both, 202 bp insertion-deletion and Friesian haplotype. The distribution of genotypes of the two putative POLLED alleles in large semi-random sample (1,261 animals) supports the hypothesis of two independent mutations

Spatio-temporal analysis of blood perfusion by imaging photoplethysmography

Imaging photoplethysmography (iPPG) has attracted much attention over the last years. The vast majority of works focuses on methods to reliably extract the heart rate from videos. Only a few works addressed iPPGs ability to exploit spatio-temporal perfusion pattern to derive further diagnostic statements. This work directs at the spatio-temporal analysis of blood perfusion from videos. We present a novel algorithm that bases on the two-dimensional representation of the blood pulsation (perfusion map). The basic idea behind the proposed algorithm consists of a pairwise estimation of time delays between photoplethysmographic signals of spatially separated regions. The probabilistic approach yields a parameter denoted as perfusion speed. We compare the perfusion speed versus two parameters, which assess the strength of blood pulsation (perfusion strength and signal to noise ratio). Preliminary results using video data with different physiological stimuli (cold pressure test, cold face test) show that all measures are in fluenced by those stimuli (some of them with statistical certainty). The perfusion speed turned out to be more sensitive than the other measures in some cases. However, our results also show that the intraindividual stability and interindividual comparability of all used measures remain critical points. This work proves the general feasibility of employing the perfusion speed as novel iPPG quantity. Future studies will address open points like the handling of ballistocardiographic effects and will try to deepen the understanding of the predominant physiological mechanisms and their relation to the algorithmic performance

cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM

Expensive scientific camera hardware is amongst the main cost factors in modern, high-performance microscopes. Recent technological advantages have, however, yielded consumer-grade camera devices that can provide surprisingly good performance. The camera sensors of smartphones in particular have benefited of this development. Combined with computing power and due to their ubiquity, smartphones provide a fantastic opportunity for "imaging on a budget". Here we show that a consumer cellphone is capable even of optical super-resolution imaging by (direct) Stochastic Optical Reconstruction Microscopy (dSTORM), achieving optical resolution better than 80 nm. In addition to the use of standard reconstruction algorithms, we investigated an approach by a trained image-to-image generative adversarial network (GAN). This not only serves as a versatile technique to reconstruct video sequences under conditions where traditional algorithms provide sub-optimal localization performance, but also allows processing directly on the smartphone. We believe that "cellSTORM" paves the way for affordable super-resolution microscopy suitable for research and education, expanding access to cutting edge research to a large community

Drawing Graphs with Circular Arcs and Right-Angle Crossings

In a RAC drawing of a graph, vertices are represented by points in the plane, adjacent vertices are connected by line segments, and crossings must form right angles. Graphs that admit such drawings are RAC graphs. RAC graphs are beyond-planar graphs and have been studied extensively. In particular, it is known that a RAC graph with n vertices has at most 4n - 10 edges. We introduce a superclass of RAC graphs, which we call arc-RAC graphs. A graph is arc-RAC if it admits a drawing where edges are represented by circular arcs and crossings form right angles. We provide a Tur\'an-type result showing that an arc-RAC graph with n vertices has at most 14n - 12 edges and that there are n-vertex arc-RAC graphs with 4.5n - o(n) edges

Optical properties of coupled metal-semiconductor and metal-molecule nanocrystal complexes: the role of multipole effects

We investigate theoretically the effects of interaction between an optical dipole (semiconductor quantum dot or molecule) and metal nanoparticles. The calculated absorption spectra of hybrid structures demonstrate strong effects of interference coming from the exciton-plasmon coupling. In particular, the absorption spectra acquire characteristic asymmetric lineshapes and strong anti-resonances. We present here an exact solution of the problem beyond the dipole approximation and find that the multipole treatment of the interaction is crucial for the understanding of strongly-interacting exciton-plasmon nano-systems. Interestingly, the visibility of the exciton resonance becomes greatly enhanced for small inter-particle distances due to the interference phenomenon, multipole effects, and electromagnetic enhancement. We find that the destructive interference is particularly strong. Using our exact theory, we show that the interference effects can be observed experimentally even in the exciting systems at room temperature.Comment: 9 page